DOCUMENT RESUME 



ED 079 066 



SE 015 920 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 

PUB DATE 
NOTE 



EDRS PRICE 
DESCRIPTORS 



Allen r David W. ^ 
An Evaluation of a Museum Science Education Project 
with social Integration and Urban Environmental 
Problems as a Focus 
Philadelphia School District^ Pa. 

Office of Education (DHEW) , Washington, D.C* Div. of 
Compensatory Education. 
Mar 73 

20p.; Paper presented at the annual meeting of the 
National Association for* Research in 'Science Teaching 
(46th, Detroit, Michigan, March 1973) 

MF-*0.65 HC-$3.29 . ' — - - 

Biology; Educational Research; Elementary School 
Science;^ ♦Instructiion; ♦Museums; Physical Sciences; 
^ ♦Program Evaluation; Science Education; ♦Social 

„\ ' Integration; ♦Student Attitudes; Urban Environment 

IDENTIFIERS Research Reports . ^ 

ABSTRACT ' ' 

The Paired School Science Project (PSSP) conducted 
between public and parochiar schqjails was evaluated over a period from 
i968771«-A pilot program was carried out among fifth and sixth grade 
students of 30 schools in the 1967-68 school year. During the study 
periodr these classes of the paired schools were bused to the 
Franklin Institute for integrated £>hysical-biological science 
instruction one day a week for seven weeks. . Students ate lunch 
together and then left for their respective schools. The program was 
€tktended to a whole day experience beginning in 1970, and a workshop 
for teachers and principals was added during the last schopl year. 
Museum resources and examination of urban environmental problems i|iere 
Emphasized. The student interaction category system, cultural 
awareness questionnaires, achievement vests, semantic differentials, 
sociometric studies, and interview, techniques were used in 
evaluation. A parent questionnaire was used in the 1970^71 school 
year. Analyses showed-^that significant gain differences existed 
between the PSSP and the control groups. Parent parti cipatioji was 
fairly successful. Sensitivity to environmental problems was 
enhanced, and student interactions occurred mostly between paired 
schools from similar geographic locations. Six days of social 
interaction had little effect on student attitude changes. (CC) 
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HISTORY OF THE PAIRED SCHOOL SCIENCE PROJECT 

The Paired School Science Project (PSSP) was pilot tested during 
the" 1967-1968 school year. It involved the pairing of both public and paro- 
chial elementary schools whose fifth- and/or sixth-grak^ classes were bused 
to the Franklin Institute for integrated science instruction. This science 
'enrichn^nt program started out as a mini-course in astronomy using the 
planetarium lessons as a focus and workshops for the development of materials 
which were taXen to the home. school. Pupils from the two schools ate lunch 
together and then left for their respective schools. . There were three eight- 
week cycles involving some 30 schools during the first year. ESEA Title I 
supported PSSP with an initial grant of $16,000. - . 

During the 1968-1969 school year, the project continued to be 
funded under E^EA Title I. The curriculum centered on the physical sciences 
and the hands-on-experiences of teams of pupils from the paired schools was 
emphasized in a workshop using an:inductive science approach (e.g., see 
Appendix A) . Relevant demonstrations using Museum resources were also 
presented and the paired schools ate lunch together. During the second 
year of the project, there were three cycles in which each paired school 
attended the Franklin Institute for lesson, pne day a week for seven weeks. 
This organizational structure was kept the same for the 1969-1970 school year 
However, the project was restricted to sixth-grade classes. 

With the increased emphasis on environmental education, the proj- 
ect director, Samuel Lepow, decided in the summer of 1970 to expand the 
program to a whole day experience, previously, pupils attended only a half 
day. The afternoon was to be used to direct pupil attention to urban envi- 
ronmental problems. The major addition to the prograi. consisted of 



field trips to" examine and understand urban environmental problems. 

An outline of the present organizational structure is presented 
in Figure- 1. Some flexibility has been built in so that pupils can visit 
exhibits in the Museum. 



Morning Session _ 
9:30. a.m. 



Short Lesson - |— 

Workshop Activiry j 



Relevant Museum 
Demonstrations 



11; 30 a.m'. 



► 1. — Topics by week include 

1. Electricity 

2. Light 

3. Newton's 3rd-law 

4. Air 
,5. Water- 

- 6. Microbiology 



Lunch and Free time 
Time to tour Museum 



Afternoon Session 
' 1:00 p.m.- 



Field Trips 



2:30 p.m. 



1, Phiia. Art Museum 

2. Bus tour to see 
major source of 
pollution 

3. Lindenwald High 
Speed Line 

4. Air Monitoring 
Station 

5. Water Works ■ 

6. Academy of 
Natural Science 



FIGUKE I 



AN OUTLINE OF THE TYPICAL SCHEDVi-E FOR ONE DAY IH^ THE 
PAIRED SCHOOL SCIENCE PROJECT 

In order to facilitate the smooth implementation of each cycle/ 
a workshop followed by a dinner is held for the participating teachers and 
principals from the paired schoolsl Teachers arid principals are asked to 
form a circuit using a wire, a battery, and a 1^5 volt bulb as part of the 
workshop. They are also given written materials which summarize the siK 
lessons and a demonstration book to supplement the science instruction at 
the Franklin Institute back at the home school- This inservice^ component 
was added during zhe current year. 



Alj EVALUATXOK MODE L 

Before discussing the evaluation of project objectives, a 
paradigm of the evaluation process (Diamond and Fishman, 1973) will be 
developed. This model of evaluation can be applied to any stage of the 
project, including the proposal. This model assumes that the evaluator 
is an integral component of the project and not an outside agent called, 
in after the fact to make judgments on the achieve^ient of project goals. 

Ik - ... - ' V* ' 



EDUCATIONAL 
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PROPOSAL 



» ASSESSMENT^ OF PROPOSAL ■ 



4 



IMPLEMENTATION 



FORMATIVE EVALUATION 

X 



X 



* FEEDBACK TO STAFF 



SUMMATIVE EVALUATI(»I 



I DISSEMINATION 
Figure 2 

■ ' PARMIGM OF THE EVALOATION PROCESS 

\ 

The evaluator should ideally be involved with, the proposal stage 
of the project. In the proposal the role of the evaluator can be cl*arly 
defined and measurable objectives can be spelled out. If this is not the 
case, a reassessment of the proposal using monitoring data from feedback 
loop is necessary to make modifications. Some of the activities which the 
evaluator should be doing during the formative evaluation include clearer 
definition of project goals (e.g., behavior objectives), assessment of 
priorities, progress reports to staff, and proposal revisions. 



The formative evaluation may be viewed as a check on the 
fidelity of th# project to its proposal." Discrepancies between the , 
proposal and the project are brought to the attention of the project 
director. 

During the implementation stage, monitoring .of zhe project 
is conducted to provide a continuous appraisal of progress in meeting 
objectives andgoals, as well as a detailed description of events ^that . 

impinge upon the project. . • • 

In the summative stage, quantifiable measures of the objectives 
are presented and compared with the baseline data, prior to the i«pi«nenta- 
tion of the project. Reports of both the formative and summative evalua- 
tions are written up and disseminated. 

EVALUATION FINDINGS 

> 

During the 1967-1968 school year, three objectives for the 
project were defined (Solomon and Brown, 1969). Instruments were designed 
to measure whether significant improvement in intercultural awareness 
(i.e., predisposition of an individual to associate with pupil from other 
school) , science achievement in astronomy and attitudes toward school had 
occurred during the eight days at the Franklin Institute. 

The three instruments consisted of (1) a cultural awareness 
questionnaire, (2) a forty question multiple choice achievement test and 
(3) a semantic differential. A control group design was used in which 
pupils who did noc participate in PSSP were compared with pupils from the 
sa.:.e school who did on instruments two and three. 
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The pupils* names from the other school were the responses which 
the cultural awareness questionnaire was to illicit. However, 99 percent 
of the respondents* choices were from the respondents' own class. Thus, 
the objective of cultural awareness, as operationally defined by the instru- 
ment, was not measured. The instrument, or ?SSP, or a combination of both, 
could be factors which led to the lack of the desired resp'3ses. 

Two alternate forms of the cognitive test of astronomy achieve- 
ment indicated that those who received the PSSP treatment had significantly 
higher scores than the control group. Using social class in an ANCOVA as 
another variable which may have effected achievement, no significance was ^ 
shown. Thus, socioeconomic status of an individual has little or no effect 
on his achievement in PSSP. ^ 

The results on the semantic differential indicate that pupils* 
opinions toward the Franklin Institute science and astronomy, were not 
different from those of the control group. This may have occiirred as _a 
result*^ interaction between the experimental and control groups back at 
the 'home school, or to lack of sensitivity of the instrument. 

During the 1968-1969 school year, the following two objectives 
were evaluated (Davidoff, 1970): 

1. To promote the understanding of basic concepts of physical 
sciences including the pupils* ability to: 

a. Recall ba^ic factual information dealing with the prin- 
ciples of matter and energy. « 

b. Define basic concepts and give examples. 

c. Understand the concept of energy and its conversions. 

2. To develop positive working relationships wxth paired school 
classmates of different ethnic backgrounds as evidenced by: 



a. cooperative g3?eup work • 

b. Sharing of equipment and ideas- 

c. Consideration of others. 

TWO null hypotheses were tested using a Student Interaction 
Category System and a specially prepared science test. ^ 

H Students V7ill spend the inajor portion of their time in 
* .01.. . ■ . . - . ■ 

' - ' goal directed activities, as measured by. the Student inter- 
action Analysis Scale (Prototype) . 

■ H There are no significant differences (p<.05) in science 
O2. 

achievement between students who participate in the Paired 
school Science Project and control groups as measured by 
a locally produced Science Achievement Test. 
Since the Student Interaction Category System <Davidof f™ 196?,) 
was a protype instrument, any conclusions drawn from the instrument are 
tentative. Null hypothesis one was rejected at the .01 level, indicating 
that there" wa^' a significant percentage of goal-oriented behavior as com- 
pared with goal-disruptive behavior. The ratio of goal-oriented behaviors 
to goal -disruptive behaviors was 6.5 to 1. 

It appears that PSSP does provide the type of environment 
suggested by Allport (1954) for reducing .prejudice by establishing equal 
status contacts between pupils of differing ethnic background in the pur- 
suit of common goals in the science lesson. 

Null hypothesis two was rejected for both the public and non- 
public schools at the .01 level, indicating that the PSSP experimental 
group learned more science than the control group. Using ANOVA with block- 
ing on pretest science scores, the statistics indicated that children 
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gained, according to their initial knowledge level. 

The d-isign evaluating the cognitive objective, H^^, was kept 
the, same for the 1969-1970 evaluation. A new instrumem;. Three American 
Twins on A Bus (Davidoff 1970, and Singer, 1966) was developed to measure 
objective- two." However,, due to its experiir.encal nature, no conclusions 
were made on the obtainment of objective -two. This instrument has been 
revised and will be mentioned in the review of the 1971-1972 findings. 

The findings on. the iognitive objective replicated the previous 
two year's results in that H was rejected. In a sense, one would expect 
children who receive the special Franklin Instilftte science lessons- to have 
higher scores on the science measure than tTiose who did not. The findings 
'in the third year may be more generalizable in that random selection pro- 
cedures were used. in selecting the schools and classes which attended ^e 
Institute. Some confounding of the treatment between the experiiaental and 
control high pretest group appears to haye occurred because of the signifi- 
cant gains made by the control group. This may have resulted from the 
sharing of materials and lessons back at the home school. 

During the 1970-1971 school year, the research centered on 

answering two questions. 

1. Has "PSSP provided the conditions that are considered pre- 
requisite for the attainment of its objectives? 

2. Have PSSP pupils demonstrated knowledge and understanding 
of basic concepts of physical science? 

J Monitoring data, as presented in Table 1, indicated that question 
one could be answered in the affirmative. Systematic monitoring indicated 
consistent results; appropriate materials had been available and used; 



instructors hcd been fulfilling their specified roles appropriai;ely; and 
pupils had' been attentive during the PSSP activ^ti^e^. 



INSERT TABLE 1 



The test results, using form A of the science test as posttest, 
•indicated that significant gains occurred over the previous two year's 
! nonparticipant comparison group's mean of 13.8. 

During the 1970-1971 evaluation of PSSP, a cluster of School 
conuniiity related projects were evaluated, using a parent 'questionnaire, 
asking parents to identify projects thay knew ana/or participated in their 
child's school. PSSP ranked second (H.=1698, 71%) among ten projects for 
accurate knowledge about the existence of the project in the school. Of 
the 72 parent responses to: Do you parcicipate in the project, 11% indi- 
cated that they had. Bussing and the si»e of the project appear to be 
factors which contributed to the high level of accurate knowledge about 
the existence of the project in their child's school. • 

The evaluator, during the 1971-1972 scl^l year, asked the same 
two cjuestions as during the previous year. However, two additional ques- 
tions were added : - . - . 

3. Have changes in the PSSP program been implemented? 

4. Has a reduction in social isolation occurred as a result 
of the pairing of schools with pupils from different ethnic backgrounds? 

The data co-llected through systematic monitoring of PSSP indi- 
cated that the morning prerequisite activities and conditions Were present 
for achieving objective 1. This data is presented in Table 1. 
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Due to rhe addition and dropping of workshop lessons ,and the I 
field trip program, a revision of the science achievement test was under- 
taken. After devexoping an item pool of some 60 items, two tests, similar 
in context, were designed. To reduce reading dependency, pictures were, 
used when appropriate. Form A and B of the Franklin Institute Science 
Quiz .were administered as- alternate forms to be used in a pre-posttest • 

design. . , 

A t.test of the difference between the pretest and the posttest 
scores indicated that a significant gain was made by PSSP pupils. The 
results of this analysis are presented in Table 2. 

INSERT TABLE 2 

The next question to be investigated was whether the proposal 
modification had been implanented. Were the field trips dealing with 
environmental education implemented? Monitoring data to answer this 
question are summarized in Table 3. Failure to achieve all the desired 
conditions indicated some difficulty in modifying PSSP to add an environ- 
mental education component. Parent participation in the program was , 
fairly successful and the field trips did occur, as planned. However, 
the communication of a problem orientation to the field trip was not 
consistently present and the ecological considerations of the problems 
of the urban environment were not incorporated into the lessons. 
Sensitivity to the urban environmental probl^s- may have been the most 
meaningful outcome of .these field trips, as measured hy the test and 
pupil interviews. 



In answering question four, as to whether a reduction in social 
ifiolation had occurred in PSSP, a sociometric study was under-taken usinc 
two approaches: (D direct observation, and <2) a paper arid pencil instru- 
ment. * 

"The direct obseirvations involved observing whether a pupil 
talked to a classmate from same school or other.' From tallies, a percent- 
age of time talking to members from the other school was calculated < The 
results are presented in Table 4. 

INSERT TABLE 4 ' ' ^ 



Once the strangeness of the new situation wore off, pupils intern- 
acted' with pupils from either school on an approximately equal basis. How 
ever, at the end of the cycle, pupil interaction 2>etween schools was re-' 
duced to the level found at the beginning of the cycle # except for schools 
two and three. This pairing of schools {i.e., C and P) which had the best 
interactions, was a nonpublic v/ith a public school from the same geographic 
local in the city. A similar observation had been made in the 1967--i968 
evaluation. Thus, a higher priority should be given to pairing of schools 
from similar geographic locations if the integration objective can also be 
fulfilled. 

The paper and pencil instrument was developed from the Three 
American Twins on a Bus sociometric instriiment (Davidoff, 1970) and was 
entitled Six Americam Twins on a Bus. Boy sind girl twins of the black, 
Spanish speaking, and white communities are pictured as going on a field 
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trip on a school bus. Pupils are asked who they would like zo do various 
activities with (association) , who they perceived as being well behaved 
(nonaggressive) , and who they perceived as achieving success in school 
(achievement) . Results of the instriiitient in a pre-to posttest design 
indicate little change of perceptions on the three scales by the chree 
races (see Table 5) . 



INSERT TABLE 5 



Tentative conclusions suggest that six days of social inter- 
action had little effect on the attitudes and feelings which go into 
choosing a friend (association) , viewing exemplary behavior (nonaggression) 
and seeing fellow pupils as successful in school (achievement) . 

In summarizing the findings on the social goals of PSSProne 
can say that meaningful interactions between pupils from different ethnic 
backgrounds are occurring. The equal status contacts between pupils, 
directed to accomplishing certain goals, {e.g., lighting a light _bulb) 
is a facilitator. However, measurable attitude changes do not appear to 
be a result of the six days of social interaction. 

The cognitive goals of learning more science are being achieved 
by PSSP. Test results indicate that the concepts ot the physical and 
biological sciences are being communicated. 
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TABLE 1 



SUMMARY OF OBSERVATIONS MADE DURING 21 VISITS MADE 
IN 1971 AND 37 VISITS MADE IN 1972 TO PSSP 



Desired Condition 



Number of Observation Visits 



Scheduled topic was 
being discussed 

Science materials 
\^ere available 

Pu?pils were construct- 
ing or working with 
science materials 

Pupils M^ed materials 
to. solve problems 

Oral instruction at 
pupils' level " 

There was a 
demonstration related 
tc the topic of the 
day 

Pupils were attentive 
tc the demonstration 



Condition 

Present 
1971 1972 



Condition 
Lacking " 
1971 1972 



; Condition not 

Appropriate 
i during 
! observation 
! 1971 1972 




TABLE 2 



ANALYSIS OF PUPIL COGNITIVE SCORES ON FORMS A AND B 
OF THi2 FRANKLIN INSTITUTE SCIENCE QUIZ 



Testing Period ' 


Pretest 


Posttest 


N 


320 




321 


Mean 


11.88 




14.24 


S.D. 


3.97 




4.89 


Mean difference 




2.36 




t value 




6.94 




P 




.01 





/ 



ERIC 



TABLE 3 

SUMMARY OF CHANGES IN PSSP OBSERVED DURING 37 VISITS 



Number of Observation Visits 


pesired Condition 


Condition 
Present 


Condition 
Lacking 


Condition not 
Appropriate 

during 
Observation 




1. Parencs present. 


15 


20 


2 ' 




2* Scheduled field trip. 


12 


0 


25 




3. Alternate seating on 
bus . 


2 


5 


30 

* 




4. Problem solving orien- 
tation to field trip. 


4 


6 


27 




5, Pupils attentive dur- 
ing field trip. 


9 


l" 


27 




6. Pupils thought the 

field trip was educa- 
tional. 


7 


" .1 


29 


7. Ecology was one topic 
mentioned. 


11 


10 


16 





TABLE 4 

PSSP SOCICr^^TRIC i:^TERACTIOK£ 



School 



percent of Interaction of Pupils from other 
School {20 Minute Time Intervals) 



Time 1 Time 2 Time 3 Time 4 Time 5 



School A and School E 
School C and School D 
School E and School F 



35% 



35% 



39% 



46% 



NOT ^OT 38% 

OBSERVED OBSERVED 



NOT 70% 
OBSERVED 



42% 



43% 



56% 



*Too few cases to calculate a meaningful percentage. 



75% 



39% 



TABLE 5 

ANALYSIS OF ATTITUDE CHANGE USING THE INSTRUMENT 
"SIX AMERICAN TWINS ON A BUS"- 
AS A PRETEST AND POSTTEST 



scales 



Achievement 



Nonaggression 



Achievement 



Pretest Mean 



Posttest Mean 



Black 




4.45 


4.88 


Puerto 


Rican 


6.29 


6.17 


White 




4.61 


. 5.00 


Black 




7.35 


7.32 


Puerto 


Rican 


5.59 


4.76 


White 




5.31 


5.72 


Black 




5.50 


5.25 


Puerto 


Rican 


6.06 


6.82 


White 




5.76 


5.80 



Overall 



Black 

Puerto Rican 
White 



17.30 
17.94 
15.68 



17.45 
17.76 
16.52 



}J 
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